Online optimal and adaptive integral tracking control for varying discrete‐time systems using reinforcement learning

Conventional closed‐form solution to the optimal control problem using optimal control theory is only available under the assumption that there are known system dynamics/models described as differential equations. Without such models, reinforcement learning (RL) as a candidate technique has been successfully applied to iteratively solve the optimal control problem for unknown or varying systems. For the optimal tracking control problem, existing RL techniques in the literature assume either the use of a predetermined feedforward input for the tracking control, restrictive assumptions on the reference model dynamics, or discounted tracking costs. Furthermore, by using discounted tracking costs, zero steady‐state error cannot be guaranteed by the existing RL methods. This article therefore presents an optimal online RL tracking control framework for discrete‐time (DT) systems, which does not impose any restrictive assumptions of the existing methods and equally guarantees zero steady‐state tracking error. This is achieved by augmenting the original system dynamics with the integral of the error between the reference inputs and the tracked outputs for use in the online RL framework. It is further shown that the resulting value function for the DT linear quadratic tracker using the augmented formulation with integral control is also quadratic. This enables the development of Bellman equations, which use only the system measurements to solve the corresponding DT algebraic Riccati equation and obtain the optimal tracking control inputs online. Two RL strategies are thereafter proposed based on both the value function approximation and the Q‐learning along with bounds on excitation for the convergence of the parameter estimates. Simulation case studies show the effectiveness of the proposed approach

Unhealthy weight control behaviours in adolescent girls: a process model based on self-determination theory

This study used self-determination theory (Deci, E.L., & Ryan, R.M. (2000). The 'what' and 'why' of goal pursuits: Human needs and the self-determination of behavior. Psychological Inquiry, 11, 227-268.) to examine predictors of body image concerns and unhealthy weight control behaviours in a sample of 350 Greek adolescent girls. A process model was tested which proposed that perceptions of parental autonomy support and two life goals (health and image) would predict adolescents' degree of satisfaction of their basic psychological needs. In turn, psychological need satisfaction was hypothesised to negatively predict body image concerns (i.e. drive for thinness and body dissatisfaction) and, indirectly, unhealthy weight control behaviours. The predictions of the model were largely supported indicating that parental autonomy support and adaptive life goals can indirectly impact upon the extent to which female adolescents engage in unhealthy weight control behaviours via facilitating the latter's psychological need satisfaction

Finite Schur filtration dimension for modules over an algebra with Schur filtration

Let G be GL_N or SL_N as reductive linear algebraic group over a field k of positive characteristic p. We prove several results that were previously established only when N 2^N. Let G act rationally on a finitely generated commutative k-algebra A. Assume that A as a G-module has a good filtration or a Schur filtration. Let M be a noetherian A-module with compatible G action. Then M has finite good/Schur filtration dimension, so that there are at most finitely many nonzero H^i(G,M). Moreover these H^i(G,M) are noetherian modules over the ring of invariants A^G. Our main tool is a resolution involving Schur functors of the ideal of the diagonal in a product of Grassmannians.Comment: 22 pages; final versio

The p53HMM algorithm: using profile hidden markov models to detect p53-responsive genes

<p>Abstract</p> <p>Background</p> <p>A computational method (called p53HMM) is presented that utilizes Profile Hidden Markov Models (PHMMs) to estimate the relative binding affinities of putative p53 response elements (REs), both p53 single-sites and cluster-sites. These models incorporate a novel "Corresponded Baum-Welch" training algorithm that provides increased predictive power by exploiting the redundancy of information found in the repeated, palindromic p53-binding motif. The predictive accuracy of these new models are compared against other predictive models, including position specific score matrices (PSSMs, or weight matrices). We also present a new dynamic acceptance threshold, dependent upon a putative binding site's distance from the Transcription Start Site (TSS) and its estimated binding affinity. This new criteria for classifying putative p53-binding sites increases predictive accuracy by reducing the false positive rate.</p> <p>Results</p> <p>Training a Profile Hidden Markov Model with corresponding positions matching a combined-palindromic p53-binding motif creates the best p53-RE predictive model. The p53HMM algorithm is available on-line: <url>http://tools.csb.ias.edu</url></p> <p>Conclusion</p> <p>Using Profile Hidden Markov Models with training methods that exploit the redundant information of the homotetramer p53 binding site provides better predictive models than weight matrices (PSSMs). These methods may also boost performance when applied to other transcription factor binding sites.</p

A polymorphism at codon 31 of gene p21 is not associated with primary open angle glaucoma in Caucasians

BACKGROUND: Primary open angle glaucoma (POAG) is considered to be a neurodegenerative optic neuropathy, in which cell death occurs by apoptosis. p21, is an important protective component of the apoptotic pathway, regulating cellular arrest in the presence of DNA damage. An unstable or altered p21 protein could modify the cellular response to genomic injury and abolish the effect of p21. A previous study on a Chinese cohort suggested that the p21 codon 31 polymorphism may alter the state of apoptosis in glaucomatous optic neuropathy, failing to protect the ganglion cells. The aim of this study was to test the hypothesis that a p21 codon 31 polymorphism is associated with POAG on a Caucasian cohort. METHODS: 140 POAG patients and a control group of 73 healthy individuals were included in the study. All the subjects were of Caucasian origin. Genomic DNA was amplified by polymerase chain reaction, followed by enzymatic restriction fragment length polymorphism technique (PCR-RFLP). Patients and controls were genotyped for a single nucleotide polymorphism (C/A transversion) in the third base of codon 31 of p21, which leads to a serine (Ser)/arginine (Arg) substitution. RESULTS: The distribution of the genotypes in the POAG patients showed 128 (91.4%) Ser homozygotes, 10 (7.1%) Ser/Arg heterozygotes and 2 (1.5%) Arg homozygotes. In the control cohort, there were 61 (83.6%) Ser homozygotes and 12 (16.4%) Ser/Arg heterozygotes. No Arg homozygotes were present amongst the control group. Both the allelic and genotypic frequencies of the Ser or Arg residues at codon 31 were not significantly different between POAG patients and controls (Fisher's exact test, P = 0.20 for alleles and P = 0.0561 for genotypes). CONCLUSION: This study suggests that the p21 codon 31 polymorphism does not contribute to the risk of POAG in the Caucasian population

From silence to primary definer: The rise of the Intelligence lobby in the public sphere

Until the end of the Cold War the UK intelligence services were not officially acknowledged, and their personnel were banned from entering the public sphere. From 1989 the UK government began to put the intelligence services on a legal footing and to release the identity of the heads of the intelligence agencies. Since then, public engagement by the intelligence agencies has gathered pace. What this article hypothesises is that there is now, in the UK, an effective intelligence lobby of former insiders who engage in the public sphere – using on the record briefings – to counter criticism of the intelligence community and to promote a narrative and vision of what UK intelligence should do, how it is supported and how oversight is conducted. Content analysis and framing models of non-broadcast coverage of intelligence debates, focusing on the 36 months after the Snowden revelations, confirm an active and rolling lobby of current and former intelligence officials. The paper concludes that the extent of the lobby’s interventions in the public sphere is a matter for debate and possible concern

Prognostic role of p27Kip1 and apoptosis in human breast cancer

Human breast carcinoma is biologically heterogeneous, and its clinical course may vary from an indolent slowly progressive one to a course associated with rapid progression and metastatic spread. It is important to establish prognostic factors which will define subgroups of patients with low vs high risk of recurrence so as to better define the need for additional therapy. Additional characterization of the molecular make-up of breast cancer phenotypes should provide important insights into the biology of breast cancer. In the present study, we investigated apoptosis, expression of p27Kip1 and p53 retrospectively in 181 human breast cancer specimens. In addition, their relevance to the biological behaviour of breast cancer was examined. Our studies found a significant association among high histological grade, high p53, low apoptosis and low p27. Our results also demonstrated that, in human breast cancer, low levels of p27 and apoptotic index (AI) strongly correlated with the presence of lymph node metastasis and decreased patient survival. In node-negative patients, however, p27 also had prognostic value for relapse-free and overall survival in multivariate analysis. Furthermore p27 and AI had predictive value for the benefits of chemotherapy. These latter observations should prompt prospective randomized studies designed to investigate the predictive role of p27 and AI in determining who should receive chemotherapy in node-negative patients. © 1999 Cancer Research Campaig

Regression based predictor for p53 transactivation

<p>Abstract</p> <p>Background</p> <p>The p53 protein is a master regulator that controls the transcription of many genes in various pathways in response to a variety of stress signals. The extent of this regulation depends in part on the binding affinity of p53 to its response elements (REs). Traditional profile scores for p53 based on position weight matrices (PWM) are only a weak indicator of binding affinity because the level of binding also depends on various other factors such as interaction between the nucleotides and, in case of p53-REs, the extent of the spacer between the dimers.</p> <p>Results</p> <p>In the current study we introduce a novel <it>in-silico </it>predictor for p53-RE transactivation capability based on a combination of multidimensional scaling and multinomial logistic regression. Experimentally validated known p53-REs along with their transactivation capabilities are used for training. Through cross-validation studies we show that our method outperforms other existing methods. To demonstrate the utility of this method we (a) rank putative p53-REs of target genes and target microRNAs based on the predicted transactivation capability and (b) study the implication of polymorphisms overlapping p53-RE on its transactivation capability.</p> <p>Conclusion</p> <p>Taking into account both nucleotide interactions and the spacer length of p53-RE, we have created a novel <it>in-silico </it>regression-based transactivation capability predictor for p53-REs and used it to analyze validated and novel p53-REs and to predict the impact of SNPs overlapping these elements.</p

The Effect of Opioid Receptor Blockade on the Neural Processing of Thermal Stimuli

The endogenous opioid system represents one of the principal systems in the modulation of pain. This has been demonstrated in studies of placebo analgesia and stress-induced analgesia, where anti-nociceptive activity triggered by pain itself or by cognitive states is blocked by opioid antagonists. The aim of this study was to characterize the effect of opioid receptor blockade on the physiological processing of painful thermal stimulation in the absence of cognitive manipulation. We therefore measured BOLD (blood oxygen level dependent) signal responses and intensity ratings to non-painful and painful thermal stimuli in a double-blind, cross-over design using the opioid receptor antagonist naloxone. On the behavioral level, we observed an increase in intensity ratings under naloxone due mainly to a difference in the non-painful stimuli. On the neural level, painful thermal stimulation was associated with a negative BOLD signal within the pregenual anterior cingulate cortex, and this deactivation was abolished by naloxone

Proteolytic cleavage of p53 mutants in response to mismatched DNA

Interaction of p53 with mismatched DNA induces proteolytic cleavage with release of a 35-kDa protein fragment from the p53–DNA complexes. The 35-kDa cleavage product is activated for specific biochemical function(s) and may play a role in the cellular response to DNA damage (Molinari et al (1996) Oncogene13: 2077–2086; Okorokov et al (1997) EMBO J16: 6008–6017). In the present study we have asked if mutants of p53 retain the ability to undergo similar proteolytic cleavage, and compared sequence-specific ‘DNA contact’ with ‘structural’ mutants commonly found in human cancer. In addition, a series of phosphorylation site mutants were generated to investigate the possible effects of phosphorylation/dephosphorylation on the proteolytic cleavage of p53. All mutants tested bound to a mismatched DNA target in vitro. Moreover, studies in vitro and in vivo indicate that p53 mutants with intact conformational structure (as determined by immunoreactivity with PAb246 and PAb1620) retain the ability to undergo proteolytic cleavage similar, if not identical, to the wild-type p53 protein. Our results suggest that the capacity for p53 to bind mismatched DNA is independent of structural conformation of the central core domain. Proteolytic cleavage, however, is crucially dependent upon a wild-type conformation of the protein. © 1999 Cancer Research Campaig